1. Field of the Invention
This invention relates to host adapter storage ports, and more particularly to apparatus and methods for managing storage port queues in host adapter storage ports.
2. Background of the Invention
In SCSI-over-Fibre-Channel networks, host ports (acting as SCSI initiators) are typically configured with a SCSI queue depth setting. This setting establishes the total number of concurrent commands that a host port can send on each ITL nexus (ITL refers to the initiator-target-LUN tuple, which is the basis for most SCSI operations). Storage ports (acting as SCSI targets) may also be configured with a queue depth setting. This setting establishes the total number of concurrent operations that the storage port can receive and process from all host ports that it communicates with. Typical standard values for the queue depth setting may be 16 for the host port queue depth, and 2048 for the storage port queue depth. Therefore, using typical settings, a storage port could potentially support a maximum of 2048/16=128 logical volumes.
Given that some modern storage controllers support up to 64K volumes, these storage controllers would theoretically require 64K/128=512 storage ports, or 1024 storage ports with redundant links. However, these same modern storage controllers may only support a maximum of 128 ports. This means it is impossible to utilize all volumes supported by the storage controller without significantly oversubscribing the storage port queues.
Problems may arise on oversubscribed storage port queues if the port receives more commands than it can queue. This is usually not the case, since I/O workload is typically bursty by nature and it is improbable that enough volumes will be accessed concurrently to cause a problem. However, a queue overrun condition will occur in the rare event that the storage port receives more commands than it can queue. When this happens, there are several options to handle the queue overrun condition.
The first option is to do nothing. This can lead to lost commands, host transport errors, performance issues caused by timeouts on the host, and general system instability. A second option is to limit the host configuration in a way that prevents overrunning the storage port queue. This is not a viable option since it can make a storage controller uncompetitive, and some hosts may already have configurations that oversubscribe the storage port queues. A third option is to change the storage controller architecture (e.g., add ports, decrease the number of supported logical volumes, etc.). This may work for future hardware designs but may not be a solution for current storage controllers. It also has the potential drawbacks of option two. Furthermore, it may be impossible or prohibitively expensive using current technology to increase the number of physical ports on the storage controller.
A fourth option is to adjust the host's global queue depth setting. Setting this to one, for example, would allow 64K volumes to be accessed on the storage controller. This is not a viable option since dramatically lowering the host queue depth will significantly degrade system performance. Furthermore, some workloads require high I/O concurrency and this option would not support such workloads. A fifth option is to return SCSI QFULL or BUSY status to a host when the storage port queue is full. This is the general solution implemented in the storage industry today. This can inform the host that the storage port queue is full, thereby allowing the host to throttle back the workload. However, one drawback to this approach is that it may cause multiple hosts to compete for the same slot or slots in the queue. This may cause a similar result to option one. Another possible outcome of this option is that it will starve slower or unlucky hosts from completing work. That is, this option may cause a host's work to receive QFULL/BUSY status so many times that it may cause the host to lose access to the storage port.
In view of the foregoing, what is needed is a solution that will enable oversubscribing storage port queues while simultaneously reducing or eliminating storage port queue overruns and/or host workload starvation. Ideally, the solution will support high I/O concurrency for high demand host workloads.